3D Shape Segmentation via Shape Fully Convolutional Networks


Pengyu Wang*, Yuan Gan*, Panpan Shui, Fenggen Yu, Yan Zhang, Songle Chen, Zhengxing Sun

Accepted to Computers & Graphics



We design a novel fully convolutional network architecture for shapes, denoted by Shape Fully Convolutional Networks (SFCN). 3D shapes are represented as graph structures in the SFCN architecture, based on novel graph convolution and pooling operations, which are similar to convolution and pooling operations used on images. Meanwhile, to build our SFCN architecture in the original image segmentation fully convolutional network (FCN) architecture, we also design and implement a generating operation with bridging function. This ensures that the convolution and pooling operation we have designed can be successfully applied in the original FCN architecture. In this paper, we also present a new shape segmentation approach based on SFCN. Furthermore, we allow more general and challenging input, such as mixed datasets of different categories of shapes which can prove the ability of our generalisation. In our approach, SFCNs are trained triangles-to-triangles by using three low-level geometric features as input. Finally, the feature voting-based multi-label graph cuts is adopted to optimise the segmentation results obtained by SFCN prediction. The experiment results show that our method can effectively learn and predict mixed shape datasets of either similar or different characteristics, and achieve excellent segmentation results.

Overview




The pipeline of our method. It may be divided into 3 stages: training process, using the SFCN architecture to train under three different features; testing process, predicting the test sets through the SFCN architecture; optimisation process, optimising the segmentation results by the voting-based multi-label graph cuts method to obtain the final segmentation results

Evaluation


we first tested 3 existing large datasets from COSEG, including the chair dataset of 400 shapes, the vase dataset of 300 shapes and the alien dataset of 200 shapes.





More Results


Our segmentation results on the Princeton Segmentation Benchmark (PSB) dataset.



The Segmentation Results of Mixed Datasets of Similar Shapes. (a) Part of the shapes in the training set; (b) Segmentation results of part of the shapes in the testing set.




The Segmentation Results of Mixed Datasets of Different Shapes. (a) Part of the shapes in the training set; (b) Segmentation results of part of the shapes in the testing set.



More results of our method.


PAPER / CODE



Citation


BibTeX format: 
@article{DBLP:journals/cg/WangGSYZCS18,
  author    = {Pengyu Wang and
               Yuan Gan and
               Panpan Shui and
               Fenggen Yu and
               Yan Zhang and
               Songle Chen and
               Zhengxing Sun},
  title     = {3D shape segmentation via shape fully convolutional networks},
  journal   = {Computers {\&} Graphics},
  volume    = {70},
  pages     = {128--139},
  year      = {2018},
  url       = {https://doi.org/10.1016/j.cag.2017.07.030},
  doi       = {10.1016/j.cag.2017.07.030},
  timestamp = {Fri, 19 Jan 2018 13:57:17 +0100},
  biburl    = {https://dblp.org/rec/bib/journals/cg/WangGSYZCS18},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}